Search for: All records

Estimating the unknown reward functions driving agents' behavior is a central challenge in inverse games and reinforcement learning. This paper introduces a unified framework for reward function recovery in two-player zero-sum matrix games and Markov games with entropy regularization. Given observed player strategies and actions, we aim to reconstruct the underlying reward functions. This task is challenging due to the inherent ambiguity of inverse problems, the non-uniqueness of feasible rewards, and limited observational data coverage. To address these challenges, we establish reward function identifiability using the quantal response equilibrium (QRE) under linear assumptions. Building on this theoretical foundation, we propose an algorithm to learn reward from observed actions, designed to capture all plausible reward parameters by constructing confidence sets. Our algorithm works in both static and dynamic settings and is adaptable to incorporate other methods, such as Maximum Likelihood Estimation (MLE). We provide strong theoretical guarantees for the reliability and sample-efficiency of our algorithm. Empirical results demonstrate the framework’s effectiveness in accurately recovering reward functions across various scenarios, offering new insights into decision-making in competitive environments. 

Abstract A multi-agency succession of field campaigns was conducted in southeastern Texas during July 2021 through October 2022 to study the complex interactions of aerosols, clouds and air pollution in the coastal urban environment. As part of the Tracking Aerosol Convection interactions Experiment (TRACER), the TRACER- Air Quality (TAQ) campaign the Experiment of Sea Breeze Convection, Aerosols, Precipitation and Environment (ESCAPE) and the Convective Cloud Urban Boundary Layer Experiment (CUBE), a combination of ground-based supersites and mobile laboratories, shipborne measurements and aircraft-based instrumentation were deployed. These diverse platforms collected high-resolution data to characterize the aerosol microphysics and chemistry, cloud and precipitation micro- and macro-physical properties, environmental thermodynamics and air quality-relevant constituents that are being used in follow-on analysis and modeling activities. We present the overall deployment setups, a summary of the campaign conditions and a sampling of early research results related to: (a) aerosol precursors in the urban environment, (b) influences of local meteorology on air pollution, (c) detailed observations of the sea breeze circulation, (d) retrieved supersaturation in convective updrafts, (e) characterizing the convective updraft lifecycle, (f) variability in lightning characteristics of convective storms and (g) urban influences on surface energy fluxes. The work concludes with discussion of future research activities highlighted by the TRACER model-intercomparison project to explore the representation of aerosol-convective interactions in high-resolution simulations.